Bidi browserstack executor http - #119
Conversation
…TTP/S in BiDi sessions In BiDi sessions, browser.execute() routes over WebSocket directly to the browser, bypassing BrowserStack's HTTP hub, so browserstack_executor: commands fail silently. Overwrite the execute command in BiDi sessions to route executor-prefixed scripts through executeScript (which always uses HTTP/S), leaving all other scripts untouched. Handles single-browser and multiremote setups. Ported from webdriverio/webdriverio#15216. Co-Authored-By: RohanImmanuel <RohanImmanuel@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…http fix(browserstack-service): route browserstack_executor commands via HTTP/S in BiDi sessions
|
🔴 SDK PR Review gate is red. Pending:
It turns green once the latest SDK PR Review Agent run reports GTG on the current head commit. A native reviewer approval is separately required by branch protection before merge. |
|
🔴 SDK PR Review gate is red. Pending:
It turns green once the latest SDK PR Review Agent run reports GTG on the current head commit. A native reviewer approval is separately required by branch protection before merge. |
SDK PR Review — 🔴 Fix 2 blocking issuesReviewed Blocking1. The Per SH-12, a feature that cannot run must disable itself loudly and leave the test unaffected — never half-enable. - try {
- if (this._browser.isMultiremote) {
- const multiRemoteBrowser = this._browser as unknown as WebdriverIO.MultiRemoteBrowser
- Object.keys(this._caps).forEach((browserName) => {
- this._routeBidiExecutorToHttp(multiRemoteBrowser.getInstance(browserName))
- })
- } else {
- this._routeBidiExecutorToHttp(this._browser as WebdriverIO.Browser)
- }
- } catch (err) {
- BStackLogger.warn(`Failed to patch execute for BiDi browserstack_executor routing; executor commands may not work in BiDi sessions: ${err}`)
- }
+ const patch = (browser: WebdriverIO.Browser, label?: string) => {
+ try {
+ this._routeBidiExecutorToHttp(browser)
+ } catch (err) {
+ BStackLogger.warn(`Failed to patch execute for BiDi browserstack_executor routing${label ? ` on ${label}` : ''}; executor commands may not work in BiDi sessions: ${err}`)
+ }
+ }
+
+ if (this._browser.isMultiremote) {
+ const multiRemoteBrowser = this._browser as unknown as WebdriverIO.MultiRemoteBrowser
+ Object.keys(this._caps).forEach((browserName) => {
+ patch(multiRemoteBrowser.getInstance(browserName), browserName)
+ })
+ } else {
+ patch(this._browser as WebdriverIO.Browser)
+ }
2. All three new tests are happy-path: single-browser BiDi patch, non-BiDi no-op, and a fully-successful 2-instance multiremote patch. None makes Non-blockingNone. Both findings above survived a falsification pass. Checked and cleared
Per-file confidence
|
|
🔴 SDK PR Review gate is red. Pending:
It turns green once the latest SDK PR Review Agent run reports GTG on the current head commit. A native reviewer approval is separately required by branch protection before merge. |
…nstance The try/catch wrapped the whole multiremote forEach, so a getInstance failure on one instance aborted the loop and left every later instance unpatched — a half-patched session indistinguishable in the logs from a fully-failed one. Wrap each instance's resolve-and-patch individually and name the failing instance in the warning. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
🔴 SDK PR Review gate is red. Pending:
It turns green once the latest SDK PR Review Agent run reports GTG on the current head commit. A native reviewer approval is separately required by branch protection before merge. |
SDK PR Review — ✅ GTGRe-reviewed at Prior findings1. Half-patched multiremote loop — RESOLVED. Confirmed non-cosmetic: reverting 2. Missing throw-path test — RESOLVED. Thunk approach
Non-blocking
CI at this head
Per-file confidence
|
|
🟢 SDK PR Review gate is green — the SDK PR Review Agent has given a GTG for this PR (the A native GitHub reviewer approval is still separately required by branch protection before this PR can merge — this check does not substitute for that. |
1 similar comment
|
🟢 SDK PR Review gate is green — the SDK PR Review Agent has given a GTG for this PR (the A native GitHub reviewer approval is still separately required by branch protection before this PR can merge — this check does not substitute for that. |
| } | ||
|
|
||
| browser.overwriteCommand('execute', async (originalExecute, script, ...args) => { | ||
| if (typeof script === 'string' && script.startsWith('browserstack_executor:')) { |
There was a problem hiding this comment.
This is the third copy of "is this an executor script?" in the package, and the only one with these semantics. The two existing ones are case-insensitive substring matches:
packages/browserstack-service/src/accessibility-handler.ts:546-555—script.toLowerCase().indexOf('browserstack_executor') !== -1packages/browserstack-service/src/cli/modules/accessibilityModule.ts:401-408— same check, duplicated
Evidence / risk: a script those two already classify as an executor call (leading whitespace, or BROWSERSTACK_EXECUTOR:) is not matched by startsWith('browserstack_executor:') here. On a BiDi session it therefore still goes out over script.callFunction and is swallowed, while the identical script keeps working on non-BiDi — a BiDi-only behaviour divergence. I could not verify whether the hub itself tolerates those variants; that determines whether this is live today or only latent.
Fix: extract a single predicate and use it in all three places, e.g. in util.ts:
export const isBrowserstackExecutorScript = (script: unknown): script is string =>
typeof script === 'string' && script.toLowerCase().includes('browserstack_executor')Question: is the case-sensitive startsWith deliberate (i.e. the hub rejects the variants)? If not, reusing the existing predicate keeps BiDi and non-BiDi behaviour identical.
| "@wdio/browserstack-service": patch | ||
| --- | ||
|
|
||
| - Fixed BrowserStack executor commands (session name, status, annotations) being ignored in WebDriver BiDi sessions. |
There was a problem hiding this comment.
This line ships to the public CHANGELOG, and two of the three things it names were not affected by the BiDi issue.
Evidence:
- Session name and status never used the executor — they go through the REST API:
_updateJob(packages/browserstack-service/src/service.ts:881) →_update(:912), aPUT/PATCHtoapi.browserstack.com. BiDi cannot affect that path. - The service's own annotate paths already use
executeScript(classic HTTP/execute/sync), so they were never swallowed either:_executeCommand(service.ts:1018-1038),AccessibilityHandler._setAnnotation(accessibility-handler.ts:603),InsightsHandler(insights-handler.ts:139).
What this PR actually fixes:
- User-written
browser.execute('browserstack_executor: …')calls — the main win. util.ts:2153 performO11ySync(reached viacli/modules/observabilityModule.ts:42on the CLI/binary path).cli/modules/accessibilityModule.ts:570 _setAnnotation(also CLI path).
Fix: reword to something like "Fixed browserstack_executor commands issued via browser.execute() being ignored in WebDriver BiDi sessions." Otherwise customers will attribute unrelated session-name/status problems to BiDi. The same wording appears in the PR description and the internal release notes.
| } | ||
|
|
||
| _routeBidiExecutorToHttp (browser: WebdriverIO.Browser) { | ||
| if (!browser.isBidi) { |
There was a problem hiding this comment.
The guard checks isBidi but not whether this is a BrowserStack session, which diverges from every other executor path in the package:
service.ts:1022(_executeCommand) —isBrowserstackSession(this._browser)accessibility-handler.ts:602,util.ts:2154,cli/modules/accessibilityModule.ts:570— same guard
The service does run against non-BrowserStack sessions (see the self-healing branch at service.ts:219, gated on !isBrowserstackSession), so on any non-BrowserStack BiDi session execute still gets overwritten and prefix-matched scripts get re-routed to /execute/sync.
Low impact in practice — nobody sends executor payloads to a non-BrowserStack grid — but it is a one-condition fix that keeps this consistent with the rest of the file:
if (!browser.isBidi || !isBrowserstackSession(browser)) {
return
}| return | ||
| } | ||
|
|
||
| browser.overwriteCommand('execute', async (originalExecute, script, ...args) => { |
There was a problem hiding this comment.
executeAsync is routed over BiDi under exactly the same condition as execute, and is left unpatched here.
Evidence — webdriverio@9.28.0, build/index.js:3534-3538:
async function executeAsync(script, ...args) {
...
if (this.isBidi && !this.isMultiremote) { // same gate as execute() at :3509
...
const result = await browser.scriptCallFunction(params);No internal caller passes an executor payload to executeAsync, so this is a user-facing gap only — a user doing browser.executeAsync('browserstack_executor: …') still gets it silently swallowed on BiDi.
Question: intentionally out of scope, or worth mirroring the same overwrite for executeAsync (here or as a follow-up)? Either is fine — flagging so it is a decision rather than an omission.
…or routing Extract the executor-script check into isBrowserstackExecutorScript in util.ts and trim leading whitespace before the prefix match, so a padded browserstack_executor: script is routed over HTTP on BiDi instead of being swallowed by script.callFunction. Kept start-anchored and case-sensitive: this is a rewrite decision, unlike the a11y shouldPatchExecuteScript substring checks which only skip a scan. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
🔴 SDK PR Review gate is red. Pending:
It turns green once the latest SDK PR Review Agent run reports GTG on the current head commit. A native reviewer approval is separately required by branch protection before merge. |
… sessions The patch only checked isBidi, so a non-BrowserStack BiDi session had execute overwritten and prefix-matched scripts rerouted to /execute/sync. Add the isBrowserstackSession guard every other executor path in the package uses. Applied per multiremote instance, so a mixed multiremote now patches only the BrowserStack leg. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
🔴 SDK PR Review gate is red. Pending:
It turns green once the latest SDK PR Review Agent run reports GTG on the current head commit. A native reviewer approval is separately required by branch protection before merge. |
…HTTP on BiDi executeAsync hits the same isBidi && !isMultiremote gate as execute in webdriverio, so a user calling browser.executeAsync with a browserstack_executor payload had it swallowed by script.callFunction. Mirror the overwrite to executeAsyncScript, which is where the classic path already lands. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
🔴 SDK PR Review gate is red. Pending:
It turns green once the latest SDK PR Review Agent run reports GTG on the current head commit. A native reviewer approval is separately required by branch protection before merge. |
|
RUN_TESTS |
07souravkunda
left a comment
There was a problem hiding this comment.
Verified the three fix commits at 283dd9a3 against the concerns raised in the inline threads — all four are addressed, and the start-anchored executor predicate is the better call over the looser substring match I originally suggested, since it keeps look-alike scripts on their normal transport.
Ran packages/browserstack-service/tests/service.test.ts at this SHA in an isolated worktree: 122 passed / 0 failed, including all seven BiDi tests. I also checked the new session gate against getCloudProvider's multiremote branch — child instances fall to the single-browser branch and match on their own options.hostname, so multiremote legs still get patched — and confirmed executeAsyncScript is the genuine protocol command for POST /session/:id/execute/async, mirroring executeScript.
The red CodeQL checks are GitHub infrastructure, not this PR: both jobs died at Set up job on 429/503 while downloading github/codeql-action, before any code was analyzed, and CodeQL passed on the three earlier runs of this branch. They need a re-run.
Two non-blocking nits left in the threads for whenever it is convenient: the BiDi tests pass the session gate only via a getCloudProvider spy leaked from describe('_update') rather than exercising the real gate, and whether the hub interprets executor payloads on /execute/async is unverified from my side — routing them there matches classic non-BiDi behaviour either way, so it does not change the verdict.
LGTM.
|
[SDK Wdio Test] TRA build state: failed | Stability 98% — verdict: success. Passed: 83, Failed: 2, Aggregate: 85. TRA: https://observability.browserstack.com/builds/d0xcobg64njqo8fhzewryh6upkaiyiz5gdnmtp6n |
What is this about?
In WebDriver BiDi sessions,
browser.execute()andbrowser.executeAsync()are dispatched over the BiDi socket (script.callFunction) instead of the classic W3C/execute/syncHTTP endpoint. BrowserStack'sbrowserstack_executor: {...}commands are interpreted by the hub on that HTTP endpoint, so an executor payload sent through either of those two commands is silently swallowed when BiDi is enabled.What was actually affected:
browser.execute('browserstack_executor: …')/browser.executeAsync(...)calls — the main case.performO11ySync(src/util.ts), reached fromcli/modules/observabilityModule.tsonBeforeTest._setAnnotationincli/modules/accessibilityModule.ts(CLI path).What was not affected, for the record: session name and status never used the executor — they go through the REST API (
_setSessionName/_updateJob→_update, a PUT/PATCH toapi.browserstack.com). The service's own annotate paths (_executeCommand,AccessibilityHandler._setAnnotation,InsightsHandler) already useexecuteScript, i.e. classic HTTP, so they were never swallowed either.This PR routes only executor payloads back over HTTP while leaving normal scripts on BiDi:
_routeBidiExecutorToHttp(browser)— no-op unlessbrowser.isBidiandisBrowserstackSession(browser), the same session guard every other executor path in the package uses. On a qualifying browser it overwritesexecute→executeScriptandexecuteAsync→executeAsyncScript; everything else falls through to the original command.isBrowserstackExecutorScript(src/util.ts): start-anchored with leading whitespace tolerated, case-sensitive — the form the hub reads and every emitter in this package produces. Deliberately narrower than the a11yshouldPatchExecuteScriptsubstring checks, which are a scan-skip heuristic where over-matching is free; here a false positive would pull a plain script off its normal transport.before(): per instance viagetInstance(browserName)for multiremote — so a mixed multiremote patches only the BrowserStack legs — and to the single browser otherwise.try/catch— a failure logs aBStackLogger.warnand the session continues rather than breaking the user's test run.Unit tests cover: executor scripts routing to
executeScript/executeAsyncScriptwhile normal scripts (and their args) pass through to the original command; leading-whitespace payloads routing while look-alike scripts do not; no overwrite on non-BiDi or non-BrowserStack sessions; and per-instance overwrite in multiremote with no cross-instance leakage.Files touched:
packages/browserstack-service/src/service.ts,packages/browserstack-service/src/util.ts,packages/browserstack-service/tests/service.test.ts.Related Jira task/s
N/A — no Jira ticket linked. Originates from #118.
Release (mandatory for every PR — required for the
ready-for-reviewlabel)Version bump: (required — tick exactly one)
Release notes type: (optional)
Release notes (customer-facing): (optional but encouraged)
browserstack_executorcommands issued throughbrowser.execute()orbrowser.executeAsync()being ignored in WebDriver BiDi sessions.Release notes (internal): (required — engineer-facing; what actually changed / why)
executeandexecuteAsyncare overwritten on BiDi browsers sobrowserstack_executor:scripts go viaexecuteScript/executeAsyncScript(classic HTTP/execute/sync,/execute/async) instead of BiDiscript.callFunction, which the hub does not intercept. Non-executor scripts still go through the original command.isBrowserstackExecutorScriptinutil.ts— start-anchored, leading whitespace tolerated, case-sensitive. Kept narrower than the a11yshouldPatchExecuteScriptsubstring checks on purpose: those decide whether to skip a scan (over-matching is free), this one rewrites transport (over-matching is not).before()per multiremote instance (getInstance) or to the single browser; skipped unlessbrowser.isBidiandisBrowserstackSession(browser), matching_update/_executeCommand/ the a11y handlers._updateJob) and theexecuteScript-based annotate paths were never affected by the BiDi dispatch and are unchanged.try/catchwith aBStackLogger.warnso a failure degrades gracefully instead of failing the session.Checklist
PR Validations
Run Tests: Comment RUN_TESTS to trigger sanity tests.